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Why GAO Did This Study 


Hubble’s continued operation 

has been dependent on manned 
servicing missions using the 
National Aeronautics and Space 
Administration’s (NASA) shuttle 
fleet. The fleet was grounded in 
early 2003 following the loss of the 
Space Shuttle Columbia, as NASA 
focused its efforts on responding to 
recommendations made by the 
Columbia Accident Investigation 
Board (CAIB). In January 2004, 
NASA announced its decision to 
cancel the final planned Hubble 
servicing mission, primarily 
because of safety concerns. 
Without some type of servicing 
mission, NASA anticipates that 
Hubble will cease to support 
scientific investigations by the end 
of the decade. 


NASA’s decision not to service 

the Hubble prompted debate 
about potential alternatives to 
prolong Hubble’s mission and 

the respective costs of these 
alternatives. This report addresses 
the basis of NASA’s cost estimates 
to (1) service Hubble using the 
shuttle and (2) implement 
recommendations made by the 
CAIB. GAO is continuing its work 
on the Subcommittee’s request that 
GAO examine the potential cost of 
a robotic servicing mission to the 
Hubble Telescope. 


What GAO Recommends 


GAO is not making 
recommendations in this report. 


www.gao.gov/cgi-bin/getrpt? GAO-05-34. 


To view the full product, including the scope 
and methodology, click on the link above. 

For more information, contact Allen Li at (202) 
512-4841 or lia@gao.gov. 
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Costs for Hubble Servicing Mission 
and Implementation of Safety 
Recommendations Not Yet Definitive 


What GAO Found 


Although a shuttle servicing mission is one of the options for servicing 
the Hubble Space Telescope, to date, NASA does not have a definitive 
estimate of the potential cost. At our request, NASA prepared an estimate 
of the funding needed for a shuttle servicing mission to the Hubble. 

NASA estimates the cost at between $1.7 billion to $2.4 billion. However, 
documentary support for portions of the estimate is insufficient. For 
example, NASA officials told us that the Hubble project’s sustaining 
engineering costs run $9 to 10 million per month, but they were unable to 
produce a calculation or documents to support the estimate because they 
do not track these costs by servicing mission. Additionally, the agency has 
acknowledged that many uncertainties, such as the lack of a design solution 
for autonomous inspection and repair of the shuttle, could change 

the estimate. 


At the same time, NASA has yet to develop a definitive cost estimate for 
implementing all of the CAIB’s recommendations but has developed a 
budget estimate for safely returning the shuttle to flight—a subset of 
activities recommended by the CAIB as needed to return the shuttle to 

full operations. NASA currently estimates return to flight costs will exceed 
$2 billion, but that estimate will likely be refined as the agency continues to 
define technical concepts. NASA provided support for portions of the 
estimate, but we found the support to be insufficient—either because key 
documents were missing or the estimates lacked sufficient detail. Further, 
NASA cautions that return to flight costs will remain uncertain until the first 
return to flight shuttle mission, which is scheduled to go to the International 
Space Station in spring 2005. 
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Source: NASA. 
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For more than a decade, the National Aeronautics and Space 
Administration’s (NASA) Hubble Space Telescope has provided unique 
images of the universe and has given scientists critical data needed to help 
understand a number of space mysteries. Hubble’s continued operation 
has been dependent on NASA’s space shuttle fleet, which carried 
crewmembers to the telescope to perform periodic maintenance and 
upgrades. The grounding of the shuttle fleet following the tragic loss of 
Space Shuttle Columbia in February 2003 put these missions to Hubble on 
hold, and in January 2004, NASA announced its decision to cancel the fifth 
and final planned Hubble servicing mission. With no future servicing 
missions, NASA anticipates that Hubble will cease to support scientific 
investigations by the end of the decade. 


NASA said that its decision to cancel the servicing mission was based 
largely on concerns about shuttle safety—specifically, the need to 
implement recommendations made by the Columbia Accident 
Investigation Board (CAIB) for safely returning the space shuttle to flight.’ 
NASA also recognized that a Hubble servicing mission would necessitate 
further changes both to shuttle hardware and operational procedures. 
However, NASA’s decision has prompted considerable debate. 
Congressional members, scientists, and space policy experts have called 
for an examination of alternatives to prolong Hubble’s mission, such as the 
possibility of servicing Hubble robotically. The CAIB Chairman suggested 
that only a study of the benefits and risks of a shuttle servicing mission to 
extend Hubble’s life could determine whether it is worth the risks. 





1 The CAIB was created by NASA to investigate the February 1, 2003, loss of the space 
shuttle Columbia. 
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Results in Brief 


This report addresses the basis for NASA’s cost estimates to (1) service 
the Hubble Telescope by using the shuttle and (2) implement the CAIB’s 
recommendations, including those recommendations directly related to 
safely returning the shuttle to flight. We are continuing our work on your 
request that we examine the potential cost of a robotic servicing mission 
to the Hubble Telescope. 


To assess the basis for NASA’s Hubble servicing mission cost estimate, 
we examined NASA’s rationale for its decision to cancel the final planned 
shuttle servicing mission, analyzed available funding estimates, and 
requested analytical and documentary support for selected high-dollar 
items to identify the sufficiency of support. In reviewing the basis for 
NASA’s cost estimate for implementing the CAIB’s recommendations, we 
analyzed available funding estimates and requested analytical and 
documentary support for selected high dollar portions of the estimate to 
determine the sufficiency of such support. We performed our review from 
March through September 2004 in accordance with generally accepted 
government auditing standards. For a complete description of our scope 
and methodology, see appendix I. 


Definitive cost estimates to facilitate decisions regarding options for 
servicing the Hubble are critical. However, NASA has not yet developed 
such an estimate for one of the options—a shuttle servicing mission. At 
our request, NASA prepared an estimate of the funding needed for a 
shuttle servicing mission to Hubble. However, NASA could not provide 
documented support for key portions of the estimate, stating that there are 
many uncertainties that could change the estimate, such as the lack of a 
design solution for two safety-related requirements—autonomous 
inspection and repair and crew rescue mission capabilities. 


Similarly, NASA does not have a definitive cost estimate for implementing 
all of the CAIB recommendations. The agency has been focusing primarily 
on those recommendations it considers necessary to return the shuttle 
fleet to flight. NASA’s current estimate for implementing those 
recommendations is more than $2 billion. NASA provided us with 
documentary support for portions of the estimate, but we found some of 
the support to be insufficient. In NASA’s view, the estimate for returning 
the shuttle fleet to flight will remain uncertain until the first shuttle 
mission to the International Space Station. 


In written comments, which are reprinted in appendix II, NASA stated that 
the agency believes that both the estimate and the methodology used in 
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Background 


calculating the costs of reinstating the servicing mission are sound and 
accurate given the level of definition of the mission at this point in time. 
However, the agency agreed that some portions of the servicing mission 
activities lacked the design maturity required to estimate the costs 
according to NASA accepted and established procedures. 


Since it was launched in 1990, the Hubble Space Telescope has sent 

back images of space that have made a significant contribution to our 
understanding of the universe. The telescope uses pointing precision, 
powerful optics, and state-of-the-art instruments to explore the visible, 
ultraviolet, and near-infrared regions of the electromagnetic spectrum. To 
keep it at the forefront of astronomical research and extend its operational 
life, Hubble’s instruments have been upgraded through a series of shuttle 
servicing missions. The fifth and final planned servicing mission was 
intended to install new science instruments, replace the telescope’s 
insulation, and replace the batteries and gyroscopes. According to NASA, 
the lifetime of the observatory on orbit is ultimately limited by battery life, 
which may extend into the 2007-2008 time frame, but scientific operations 
are limited by the gyroscopes that stabilize the telescope—whose lifetimes 
are more difficult to predict. NASA forecasts that the Hubble will likely 
have fewer than three operating gyroscopes by mid-2006, and fewer than 
two by mid-2007. 


In response to congressional concerns about NASA’s decision to cancel 
the servicing mission, NASA requested that the National Research Council’ 
conduct an independent assessment of options for extending the life of the 
Hubble Space Telescope. In May 2004, the Council established a 
committee to assess the viability of a shuttle servicing mission, evaluate 
robotic and ground operations to extend the life of the telescope as a 
valuable scientific tool, assess telescope component failures and their 
impact, and provide an overall risk-benefit assessment of servicing 
options. In an interim report issued in July 2004, the committee urged 
NASA to commit to a Hubble servicing mission that accomplishes the 


* The National Research Council is part of the National Academies, which also comprise 
the National Academy of Sciences, National Academy of Engineering, and Institute of 
Medicine. They are private, nonprofit institutions that provide science, technology, and 
health policy advice under a congressional charter. The Research Council was organized by 
the National Academy of Sciences in 1916 to associate the broad community of science and 
technology with the Academy’s purposes of further knowledge and advising the federal 
government. 
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NASA's Cost Estimate 
for a Hubble 
Servicing Mission 
Using the Shuttle Is 
Not Definitive 


objectives of the canceled servicing mission and to take no actions that 
would preclude using a space shuttle to carry out this mission. According 
to a NASA official, the agency is not actively pursuing the shuttle servicing 
option but is not precluding it. 


NASA is currently evaluating the feasibility of performing robotic 
servicing of the Hubble Telescope. To facilitate the evaluation, the agency 
has formulated a robotic mission concept, which includes a vehicle 
comprised of a robotic servicing module and another module that can 

be used to eventually de-orbit the telescope. The potential task list of 
activities for robotic servicing includes replacing the gyroscopes and 
batteries, installing new science instruments, and de-orbiting the 
observatory at the end of its life. According to a NASA official, contracts to 
facilitate the robotic mission were recently awarded for work to begin on 
October 1, 2004. 


The CAIB concluded that the Columbia accident was caused by 

both physical and organizational failures. The Board’s 15 return to flight 
recommendations necessary to implement before the shuttle fleet can 
return to flight primarily address the physical causes of the accident and 
include eliminating external tank debris shedding and developing a 
capability to inspect and make emergency repairs to the orbiter’s thermal 
protection system. NASA publishes periodic updates to its plan for 
returning the shuttle to flight to demonstrate the agency’s progress in 
implementing the CAIB recommendations. The most recent update is 
dated August 27, 2004. This update identifies the first shuttle flight as 
occurring in spring 2005. 


NASA does not currently have a definitive cost estimate for servicing the 
Hubble Telescope using the shuttle. The agency focused on safety 
concerns related to a servicing mission by the space shuttle in deciding 
not to proceed, and did not develop a cost estimate. At our request NASA 
prepared an estimate of the funding needed for a Hubble servicing mission 
by the space shuttle. NASA could not provide documented support for its 
estimate. The agency recognizes that there are many uncertainties that 
could change the estimate. NASA has now begun to explore the costs and 
benefits of various servicing alternatives, including robotic servicing, 
which should enable NASA to make a more informed decision regarding 
Hubble’s future. 
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At our request NASA began development of an estimate of the funding 
needed for a shuttle servicing mission to the Hubble. The estimate 
provided captures additional funds over and above NASA’s fiscal year 2005 
budget request that would be required to reinsert the mission in the shuttle 
flight manifest for launch in March 2007. The estimate does not include 
funding already expended to support the canceled servicing mission and 
develop the science instruments. NASA has determined that the additional 
funds needed to perform a shuttle servicing mission for Hubble would be 
in the range of $1.7 billion to $2.4 billion. According to NASA, this estimate 
is based on what it might cost, but it does not take into account the 
technical, safety, and schedule risks that could increase the cost and/or 
undermine the viability of the mission. For example, NASA cites 
uncertainties related to two safety-related requirements: inspection and 
repair and crew rescue mission capabilities that would be autonomous of 
the International Space Station and for which NASA currently has not 
formulated a design solution.’ In addition, NASA cautions that it did not 
examine whether design solutions could be accomplished in time to 
service Hubble before it ceases operations. Table 1 shows NASA’s budget 
estimate phased by fiscal year (FY) for shuttle servicing of the Hubble 
Space Telescope, including ranges for some of the estimates. 





? Although the autonomous inspection and repair capability of the shuttle’s thermal 
protection system is a CAIB requirement for all shuttle missions, the currently planned 
method depends on International Space Station assets, such as the International Space 
Station’s robotic arm to stabilize a spacewalk crew making repairs to the shuttle. A Hubble 
mission would not have this asset available; therefore, NASA would have to develop an 
alternate method for stabilizing a crewmember making repairs. The second requirement— 
NASA’s requirement for a crew rescue shuttle—would apply to all shuttle flights, but 
according to NASA, the agency would need to dedicate two shuttles to a Hubble servicing 
mission because of the shorter amount of time to react to an emergency because the crew 
would not have the benefit of the International Space Station as a safe haven. The agency 
contends that the second (rescue) shuttle for the Hubble servicing mission would need to 
be on the adjacent launch pad in countdown mode at the same time the first shuttle is 
launched. The rescue shuttle capability would require the development of a second 
generation boom to be used to transfer the crew from the stricken orbiter and an enhanced 
camera to process imagery in time to support a go-no go rescue mission decision. In 
addition, a crew return kit would be needed to provide seats on the rescue shuttle for the 
rescued crew, and a special crew would have to be trained for the rescue mission. 
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Table 1: Budget Estimate for Shuttle Servicing of Hubble 





Real year dollars in millions? 









































Fiscal year 
Budget to 

Description 2004 2005 2006 2007 2008 2009 Complete Total’ 
Sustaining engineering for 
Hubble project $115 $141 $105 $26 $30 $40 $457 
Delay de-orbit mission 
until 2012 (69) (136) 79 166 40 
Extend Hubble operations 
to 2012 117 117 
Shuttle services unique to 
servicing mission and 
extravehicular activity” 18 15 33 
Payload processing 7 7 13 10 7 43 
Autonomous inspection and 
repair capability $116-134 74-86 52-60 18-21 260-300 
Autonomous rescue mission 
capability 85-98 116-134 61-71 31-36 293-338 
Rescue mission unique 
requirements and ground 
operations 1 22 22 
International Space Station 
Program impact (3-month 
delay) (5) (15) (13) (4) 50 13 
Delay shuttle phase-out 
3 months 9-0 17-72 26-249 350-757 401-1,078 
Total’ $200-231 $312-341 $275-293 $128-127 $(95)-(41) $137-361 $723-1,130 $1,679-2,441 





Source: NASA. 


Note: Estimates are in full cost. The concept of full cost ties all agency costs (including civil service 
personnel costs) to major activities. This includes costs that are directly related to a specific project, 
such as contractor-supplied hardware, and indirect costs such as administrative costs, rents, utilities, 
and travel costs that cannot be directly identified with a specific project but which can be allocated 
based on direct labor hours, square footage, or other methods. 


“In some cases, totals do not add because of rounding. 


*Extravehicular activity is a space walk. 
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While we did not independently verify each component of NASA’s 
estimate, we requested that NASA provide the analytical basis and 
documentary support for selected portions of the estimate, primarily those 
with large dollar values. NASA could not provide the requested 
information. For example, NASA officials told us that the Hubble project’s 
sustaining engineering costs run $9 tol0 million per month, but they were 
unable to produce a calculation or documents to support the estimate 
because they do not track these costs by servicing mission. We also 
requested the basis of estimate for the costs to delay shuttle phase-out and 
for tools development for vehicle inspection and repair without the 
International Space Station (a component of extravehicular activity 
above). In response, NASA provided the assumptions upon which the 
estimates were based and stated that the estimates were based on 
information provided by Johnson Space Center and Kennedy Space Center 
subject matter experts. NASA also added that rigorous cost estimating 
techniques could not be applied to the tools development estimate 
because a rescue mission currently is only a concept. No analytical or 
documentary support was provided. In estimating the cost for the 
autonomous inspection and repair and rescue mission capabilities, NASA 
used a 30 to 50 percent uncertainty factor because of the very high 
uncertainty in the cost of developing and conducting a mission that is not 
adequately defined—i.e., NASA’s estimate of $425 million plus 50 percent 
equals the $638 million upper range shown in the table above for these two 
items added together. As with the other estimates for which we requested 
analytical and documentary support, NASA was not able to provide it 
because the agency could not do a risk analysis without a design solution, 
according to a NASA official. The lack of documented support for portions 
of NASA’s estimate increases the risk of variation to the estimate. Further, 
NASA recognizes that there are many uncertainties that could change the 
current estimate. 


The 2004 NASA Cost Estimating Handbook states that cost analysts should 
document the results of cost estimates during the entire cost estimating 
process and that the documentation should provide sufficient information 
on how the estimate was developed so that independent cost analysts 
could reproduce the estimate. According to the handbook, the value of the 
documentation and analysis is in providing an understanding of the cost 
elements so that decision-makers can make informed decisions. 
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Estimate for 
Implementing CAIB 
Recommendations Is 
Not Fully Developed 


Recently, we also reported that dependable cost estimates are essential for 
establishing priorities and making informed investment decisions in the 
face of limited budgets.* Without this knowledge, a program’s estimated 
cost could be understated and thereby subject to underfunding and cost 
overruns, putting programs at risk of being reduced in scope or requiring 
additional funding to meet their objectives. 


Since we began our review, attention has focused on alternatives to a 
shuttle mission, such as robotic servicing of Hubble. NASA has formed a 
team to evaluate Hubble servicing alternatives, including cost information. 
This analysis should enable NASA to make a more informed decision 
about Hubble’s future and facilitate NASA’s evaluation of the feasibility of 
robotic servicing options. 


Currently, NASA has developed budget estimates for implementing the 
CAIB recommendations required to return the space shuttle to flight but 
not for all of the CAIB recommendations. NASA provided us with 
documentary support for portions of the return to flight estimate, but we 
found it to be insufficient. According to NASA, the agency’s cost for 
returning the shuttle to flight, which is slightly over $2 billion, will remain 
uncertain until the completion of the first shuttle missions to the 
International Space Station in fiscal year 2005. 


NASA’s return to flight activities involve enhancing the shuttle’s external 
tank, thermal protection system, solid rocket boosters, and imagery 
system to address the physical cause of the Columbia accident—a piece of 
insulating foam that separated from the external tank and struck a 
reinforced carbon-carbon panel on the leading edge of the orbiter’s left 
wing. To address this cause, NASA is working to eliminate all external 
tank debris shedding. Efforts are also in place to improve the orbiter’s 
thermal protection system, which includes heat resistant tiles, blankets, 
and reinforced carbon-carbon panels on the leading edge of the wing and 
nose cap of the shuttle, to increase the orbiter’s ability to sustain minor 
debris damage. NASA is also redesigning the method for catching bolts 
that break apart when the external tank and solid rocket boosters separate 
as well as providing the capability to obtain and downlink images after the 
separation. NASA and the United States Air Force are working to improve 





s GAO, NASA: Lack of Disciplined Cost-Estimating Processes Hinders Effective Program 
Management, GAO-04-642 (Washington, D.C.: May 28, 2004). 
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the use of ground cameras for viewing launch activities. Table 2 shows 
NASA’s budget estimates for return-to-flight activities. 


[ag a a ee ee ran] 
Table 2: Return to Flight Budget Estimates as of July 2004 





Real year dollars in millions* 















































Fiscal year 
Activity 2003 2004 2005 2006-2009° Total” 
Orbiter reinforced carbon-carbon inspections $2 $38 $7 
On-orbit thermal protection system inspection 
and extravehicular activity tile repair 20 68 130 
Orbiter workforce 0 5 37 
Orbiter thermal protection system hardening 0 28 34 
Orbiter certification/verification 0 47 26 
Orbiter other 0 15 16 
External tank items (camera, bipod ramp, etc.) 11 114 94 
Solid rocket booster items (bolt catcher, 
camera, etc.) 8 26 
Ground camera ascent imagery upgrade 8 40 58 
Kennedy Space Center ground operations 
workforce 0 32 36 
Other (system integration, hardware processing 
and operations systems verification, and space 
shuttle main engine technical assessment) 0 67 177 
Stafford-Covey team 3 1 
Total’ $42 $465 $643 $1,067 $2,217 





Source: NASA. 


Note: According to NASA, not all elements of full cost have been distributed to return to flight 
activities. 


“Fiscal years 2006-2009 are NASA preliminary estimates, and a cost breakout by activity is not 
available. 


"In some cases, totals do not add because of rounding. 


However, the majority of NASA’s budget estimates for returning the 
shuttle to flight are not fully developed—including those for fiscal year 
2005—as indicated by the agency’s internal approval process. The Program 
Requirements Control Board (PRCB) is responsible for directing studies of 
identified problems, formulating alternative solutions, selecting the best 
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solution, and developing overall estimates.’ According to NASA, actions 
approved with PRCB directives have mature estimates, while those with 
control board actions in process—that is currently under review but with 
no issued directives yet—are less mature. Both the content and estimates 
for return to flight work that have not yet been reviewed by the control 
board are very preliminary and subject to considerable variation. Table 3 
shows the status of control board review of NASA return to flight budget 
estimates and the percent of the total estimate at each level of review. 


a Ee 
Table 3: Return to Flight Budget Estimate Review Status as of July 2004 





Real year dollars in millions 




















Fiscal year 

Percent 
Review status 2003 2004 2005 2006 2007 2008 2009 Total? of total’ 
Control board review 
complete; directive issued $31 $319 $117 $53 $47 $49 $39 $655 29 
Been to control board; 
directive not yet issued 11 146 217 117 125 134 109 859 39 
In review process 309 162 84 79 70 704 32 
Total Return to Flight 
activities? $42 $465 $643 $331 $257 $261 $218 $2,217 100 





Source: NASA. 


Note: According to NASA, not all elements of full cost have been distributed to return to flight 
activities. 


“In some cases, totals do not add because of rounding. 





5 The membership of the PRCB includes the Space Shuttle Program Manager, Deputy 
Manager, all Project and Element Managers, Safety and Mission Assurance personnel, and 
the Team Leader of the return to flight Planning Team. 
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NASA provided us with the PRCB directives and in some cases, 
attachments which the agency believes support the estimate.’ However, 
we did not find this support to be sufficient. According to NASA’s cost 
estimating handbook, estimates should be documented with sufficient 
detail to be reproducible by an independent analyst. Nevertheless, in many 
cases, there were no documents attached to the directive, and in cases 
where documents were attached to the directives, the documents 
generally provided high-level estimates with little detail and no 
documentation to show how NASA arrived at the estimate. For example, a 
request for $1.8 million to fund the network to support external tank 
camera transmissions indicated that $1.516 million of the amount would 
be needed for Goddard Space Flight Center to provide the necessary 
equipment at receiving stations, labor, subcontractor costs, and travel and 
that the remaining $290,000 would be needed for improvements to the 
receiving antennas ($104,000) and recurring costs ($62,000 per flight) for 
three trucks and the associated transponder time. However, the 
documents did not show how the requester for the $1.8 million arrived at 
the estimates. NASA officials told us that the reason for this was that the 
managers approving the directives trusted their employees to accurately 
calculate the estimate and maintain the support. In addition, our review of 
the documents indicated and NASA confirmed that quite a few of the 
estimates were based on undefinitized contract actions (UCA)—that is, 
unnegotiated contract changes. Under these actions, NASA officials can 
authorize work to begin before NASA and the contractor agree on a final 
estimated cost and fee. As we have stated in our high-risk series, relying 
on unnegotiated changes is risky because it increases the potential for 





ê We attempted to obtain analytical and documentary support for the return to flight 
estimates on two occasions. First, in May 2004, we requested support for three high-dollar 
items for which we were told the estimates ranged from strong to weak to a mixture. In 
response to this request, NASA told us that the detailed bases of estimate for the activities 
were being developed and that NASA would report on return to flight expenses to Congress 
and GAO in a few weeks. Then, several weeks later when NASA released the July 28 return 
to flight update, we requested support for the three largest estimates, two of which were 
the same items for which we requested support in May. In response, NASA provided the 
PRCB directives and backup documents underlying the estimates that the agency considers 
to be mature. After we reviewed these documents and concluded that the support was not 
adequate, NASA offered to select some additional examples to show that the estimates 
were credible. 
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unanticipated cost growth.’ This, in turn, may force the agency to divert 
scarce budget resources intended for other important programs. As of 
July 31, 2004, NASA records showed 17 UCAs related to return to flight 
with not-to-exceed amounts totaling $147.5 million. NASA’s estimate for 
the entire effort under these UCAs totals about $325 million, or 15 percent 
of NASA’s current $2.2 billion return to flight estimate. 


In June 2004, NASA established additional requirements for funding 
requests submitted to the PRCB. Under the new policy, an independent 
cost estimate must be developed for requests greater than $25 million, and 
a program-level cost evaluation must be completed for requests over 

$1 million. The program-level evaluation consists of a set of standard 
questions to document the rationale and background for cost-related 
questions. The responses to the questions are initially assessed by a cost 
analyst but are reviewed by the Space Shuttle Program Business Manager 
before submission to the PRCB. 


NASA provided us with two examples of requests falling under the new 
requirements. Both of the examples had better support than those with 
PRCB directives, but documentary support was still not apparent. For 
example, the funding request for a debris radar indicated that the estimate 
was based on a partnering agreement with the Navy and the Navy’s use of 
the technology. However, the program-level evaluation pointed out that no 
detailed cost backup was provided. The other example, which was a 
funding request to change the processes currently in place for the Space 
Shuttle Program’s problem reporting and corrective action system, was 
very well supported in terms of analysis, as the requester prepared 
detailed spreadsheets calculating the funding requirements according to a 
breakdown of the work to be performed, cited sources for labor rates, and 
provided assumptions underlying the calculations. However, as pointed 
out in the program evaluation of the request, there was no support 
provided for the estimate other than the initiator’s knowledge of the 
change. We believe that future compliance with NASA’s new policy 
establishing additional requirements for funding requests and the inclusion 


f GAO, NASA: Major Management Challenges and Program Risks, GAO-03-849T 
(Washington, D.C.: June 12, 2003); GAO, High-Risk Series: An Update, GAO-03-119 
(Washington, D.C.: January 2003); GAO, Major Management Challenges and Program 
Risks: National Aeronautics and Space Administration, GAO-03-114 (Washington, D.C.: 
January 2003); GAO, High-Risk Series: An Update, GAO-01-263 (Washington, D.C.: 
January 2001); and GAO, Major Management Challenges and Program Risks: National 
Aeronautics and Space Administration, GAO-01-258 (Washington, D.C.: January 2001). 
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Agency Comments 
and Our Evaluation 


of documentary support could potentially result in more credible return to 
flight budget estimates. 


According to NASA, estimates for fiscal year 2005 and beyond will be 
refined as the Space Shuttle Program comes to closure on return to flight 
technical solutions and the return to flight plan is finalized. NASA expects 
that by late fall of 2004, a better understanding of the fiscal year 2005 
financial situation will be developed. However, NASA cautions that the 
total cost of returning the shuttle to flight will remain uncertain until 
completion of the first shuttle missions to the space station, scheduled to 
begin in spring 2005. 


In written comments on a draft of this report, the NASA Deputy 
Administrator stated that the agency believes that both the estimate and 
methodology used in the calculation of costs for reinstating the Hubble 
Space Telescope servicing mission are sound and accurate, given the level 
of definition at this point in time. Notwithstanding that belief, the agency 
agreed that portions of the servicing mission activities lacked the design 
maturity required to estimate the costs according to accepted and 
established NASA procedures. 


Specifically, NASA agrees that the Hubble Space Telescope work 
breakdown structure was not constructed to collect program costs. At the 
same time, NASA believes it is erroneous to suggest that NASA has no 
valid basis for the numbers provided, citing the “Servicing Mission 4 
Resources Management Plan,” which describes the effort required for 
completion of a servicing mission. According to NASA, although the 
program’s accounting system does not capture sustaining engineering 
costs in GAO’s preferred format, the Servicing Mission 4 Resources 
Management Plan details mission schedules and staffing, and applying 
contractor and civil service rates to that staffing level can accurately 
reflect the effort required to execute a servicing mission. We requested 
this type of analysis and documentary support, but NASA representatives 
did not offer such a calculation. Rather, the officials stated that the 
sustaining engineering costs were based on management’s assessment of 
contractor financial data and in-house service pool charges and that these 
activities could not be traced back to source documentation. Without 
adequate supporting data, we cannot assess the accuracy and reliability of 
such information. 


NASA acknowledged that the agency does not have a technical design 
from which to derive the cost for the on-orbit inspection and repair of the 
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shuttle independent of support from the International Space Station. In the 
case of the unsupported cost estimate for delaying the phase out of the 
space shuttle in order to complete a manned Hubble servicing mission, 
NASA stated that it used approved budget projections for the operating 
years affected by the insertion of the Hubble servicing mission and 
prorated the extension of the service life. According to NASA, a range was 
added to the estimate to account for uncertainties and retention of critical 
skills. The estimates were presented as a rough order of magnitude. NASA 
stated that it provided its assumptions to demonstrate the reasonableness 
of the estimates. Nevertheless, in spite of the uncertainties in the estimate, 
which we recognized in our report, NASA guidance states that cost 
estimates should be documented during the entire cost estimating process 
and that the documentation should provide sufficient information on how 
an estimate was developed to be reproducible by independent cost 
analysts. NASA did not provide us with this type of documentation. 
Without adequate supporting data, we cannot assess the accuracy and 
reliability of such information. We do not agree that the use of approved 
budget projections is a reliable cost estimating methodology, particularly 
given the long-term budget implications of the extension of the space 
shuttle’s service life. 


NASA believes that the examples it provided of the actions to implement 
several of the CAIB recommendations attest to the rigor of the process and 
approved procedures NASA utilized to validate the costs. According to 
NASA, the estimates will mature as the technical solutions mature, but the 
estimates were not refined at the time of our review. The agency believes 
the outstanding technical issues necessary to return to flight are beginning 
to be resolved. However, the examples that NASA provided were in 
support of estimates that the agency considers mature. We requested 
support for high dollar portions of NASA’s estimate, which the agency did 
not provide. However, NASA selectively provided examples of what it 
considered to be mature estimates. We reviewed the examples but found 
that most of them contained insufficient documentation to assess the 
reliability of the estimates. In many cases, there were no documents in the 
approval packages to support the estimates, and in cases where there were 
documents, they generally provided high-level estimates with little detail 
and no documentation to show how NASA arrived at the estimates. We 
believe that because of its difficulty providing reliable cost estimates, 
NASA cannot provide the Congress assurance that its budget request for 
the shuttle program for fiscal year 2006 will be sufficient and that 
shortfalls would not need to be met through reductions in other NASA 
programs. 
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NASA stated that it believes the use of UCAs is both reasonable and 
necessary for return to flight activities. We agree that UCAs may be 
justified to facilitate work outside the scope of existing contracts to 
expedite the return to flight activities. However, the use of UCAs appears 
to be a growing trend and is a risky contract management practice because 
it increases the potential for unanticipated cost growth. In the past, we 
cited the agency’s use of UCAs as one of the reasons we retained contract 
management as a high-risk designation for NASA to focus management 
attention on problem areas that involve substantial resources.® 


Finally, NASA agrees that cost estimates for significant development 
activities should be appropriately documented. According to NASA, 
additional requirements for cost estimates and internal controls recently 
established by the program represent a step in ensuring the appropriate 
documentation is developed as solutions are identified. As stated in our 
report, we believe that future compliance with this new policy could 
potentially result in more credible budget estimates. 


In a broader context, reliable and supportable cost estimating processes 
are important tools for managing programs. Without this knowledge, a 
program’s estimated cost could be understated and thereby subject to 
underfunding and cost overruns, putting programs at risk of being reduced 
in scope or requiring additional funding to meet their objectives. Further, 
without adequate financial and nonfinancial data, programs cannot easily 
track an acquisition’s progress and assess actions to be taken before it 
incurs significant cost increases and schedule delays. 


As agreed with your offices, unless you announce its contents earlier, we 
will not distribute this report further until 30 days from its date. At that 
time, we will send copies to the NASA Administrator and interested 
congressional committees. We will make copies available to others upon 
request. In addition, the report will be available at no charge on the GAO 
Web site at http://www.gao.gov. 


8 High-Risk Series: An Update (GAO-01-263, January 2001). 
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If you or your staff have any questions concerning this report, please 
contact me at (202) 512-4841 or lia@gao.gov. Key contributors to this 
report are acknowledged in appendix III. 


llen HL 


Allen Li 
Director 
Acquisition and Sourcing Management 
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Appendix I: Scope and Methodology 


To assess the basis for NASA’s Hubble servicing mission cost estimate, we 
analyzed NASA’s estimate of the funding needed for a shuttle servicing 
mission and supporting documentation, and we reviewed NASA 
documents explaining the rationale for the decision and identifying 
alternatives to shuttle servicing. We interviewed program and project 
officials to clarify our understanding of the available cost information and 
NASA’s rationale for the decision. To test the sufficiency of the support for 
the estimates provided by NASA, we requested the analytical basis and 
documentary support for selected portions of the estimates, primarily 
those with large dollar values. In addition, we compared NASA’s decision- 
making process with relevant Office of Management and Budget and NASA 
guidance on information and analyses recommended to enable decision- 
makers to select the best alternative. 


To determine the basis for NASA’s cost estimate for implementing all of 
the CAIB recommendations, we reviewed the CAIB report (volume 1), 
NASA’s return to flight implementation plan and budget estimates, and 
agency documentation discussing the return to flight budget estimate. We 
interviewed program officials to obtain a better understanding of NASA’s 
plans for returning the space shuttle to flight, the status of that effort, and 
the estimated cost. To test the sufficiency of the support for NASA’s return 
to flight estimate, we requested the analytical basis and documentary 
support for selected high dollar portions of the estimate. 


To accomplish our work, we visited NASA Headquarters, Washington, 
D.C.; and Goddard Space Flight Center, Maryland. 


We performed our review from March through September 2004 in 
accordance with generally accepted government auditing standards. 
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Appendix IJ: Comments from the National 
Aeronautics and Space Administration 








National Aeronautics and 
Space Administration 

Office of the Administrator 
Washington, DC 20546-0001 





November 10, 2004 


Mr. Allen Li 

Director 

Acquisition and Sourcing Management 

United States Government Accountability Office 
Washington, DC 20548 


Dear Mr. Li: 


The National Aeronautics and Space Administration (NASA) appreciates the opportunity 
to comment on your draft report (Government Accountability Office (GAO) Code 120335, 
Report Number GAO-05-34) entitled “Costs for Hubble Servicing Mission and Implementation 
of Safety Recommendations Not Yet Definitive.” 


Costs for Reinstating a Shuttle-based Hubble Servicing Mission (SM) 

NASA believes that both the estimate and the methodology used in the calculation of costs 
for reinstating an SM-4 mission are sound and accurate given the level of definition of the 
mission at this point in time. It is true, as the GAO has recognized in the report, that some 
portions of the identified Shuttle SM-4 reinstatement activities lacked the design maturity 
required to estimate the costs according to NASA accepted and established procedures. NASA 
believes that the cost ranges applied to these items are reasonable given that acceptable design 
solutions are unknown at this time. Comments on specific statements contained in the report are 
offered in the following paragraphs: 





Page 7: “NASA could not provide . . . the analytical basis and documentary support for selected 
portions of the estimate.” The report goes on to cite the Hubble program’s $9-10 million 
monthly burn rate for the core engineering team, stating that NASA was “unable to produce a 
calculation or documents to support the estimate because they do not track these costs by 
servicing mission.” 


NASA Response: While it is true that the Hubble Space Telescope (HST) work breakdown 
structure was never constructed to collect program costs by individual servicing mission, it is 
erroneous to suggest that NASA has no valid basis for the numbers provided. 


Over the course of this study, NASA granted GAO full access to all HST program 
accounting archives and budget estimates, including budget packages, presentation materials, 
contractor data, and supporting documentation. Among the materials provided to GAO was a 
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Appendix II: Comments from the National Aeronautics and Space Administration 








report entitled, “Servicing Mission 4 Resources Management Plan,” dated November 2003. This 
document includes a detailed description of the effort required for safe completion of a servicing 
mission to Hubble, which NASA has done successfully four times since the observatory was 
launched in 1990. 


Though the program’s accounting system does not capture sustaining engineering costs in 
the format that GAO would have preferred, the effort involved with mission safety and success is 
well understood. The Resources Management Plan details mission schedules, historical staffing 
levels, skill mix, contractor and civil service FTEs—all of which have been refined as a result of 
experience, and lessons learned over the years. The $9-10 million per month can be derived by 
applying contractor and civil service rates to these FTE numbers. It is a simple computation that 
accurately reflects the effort required to safely execute SM-4. 


Ina letter to NASA Administrator Sean O’Keefe dated March 11, 2004, Senators 
Christopher Bond and Barbara Mikulski requested that NASA “take no action to stop, suspend or 
terminate any contracts or employment in connection with the final servicing mission until this 
study is completed and Congress has taken action on NASA’s fiscal year 2005 budget.” In 
response to this request, the Hubble program has continued to retain the core team of experts 
described in the Resources Management Plan—some of whom have worked on the project since 
launch—at the $9-10 million per month level. These are the people needed to do the job safely, 
and NASA stands by the estimates provided. 


Page 7: “We also requested the basis of estimate for the costs to delay Shuttle phase-out and for 
tools development for [on-orbit] vehicle inspection and repair without the International Space 
Station . . . In response, NASA provided the assumptions upon which the estimates were based 
... No analytical or documentary support was provided.” 


NASA Response: As acknowledged on Page 4 of the report, NASA was requested to provide an 
estimate for the delay in Shuttle phase-out and on-orbit inspection and repair, independent of 
support from the Space Station, by the GAO. The report also acknowledges that NASA has no 
baseline technical concept for the on-orbit inspection and repair of the Shuttle, independent of 
the station, upon which to base a cost estimate. In the case of the Shuttle phase-out delay costs, 
NASA simply used approved budget projections for the operating years affected by the insertion 
of a Hubble mission and prorated an extension of the Shuttle’s service life beyond the 2010 end 
point assumed in the President’s FY 2005 request. A range was added to the estimate to account 
for uncertainties associated with continued production of Shuttle hardware and retention of 
critical skills. The on-orbit inspection and repair estimate was represented at the time as a 
“rough order of magnitude (ROM)” and is consistent with the lack of any reasonable level of 
technical detail. Still, NASA provided its assumptions, along with a description of one possible 
approach and a summary of technical uncertainties, with which the reasonableness of the ROM 
estimate could be determined. 


Costs for Implementing Safety Recommendations 

NASA is committed to full implementation of all of the recommendations of the Columbia 
Accident Investigation Board (CAIB). NASA’s Return to Flight (RTF) efforts continue to 
evolve as appropriate solutions are identified. NASA employs a rigorous review process to 
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Appendix II: Comments from the National Aeronautics and Space Administration 








approve specific projects required to safely return the Shuttle to flight. The technical scope of 
work is defined, schedules are built, and standard cost-estimating methodologies are employed to 
gain approval for implementation. NASA believes that the detailed examples provided to the 
GAO attest to the rigor of the technical process and that NASA is applying approved procedures 
for validating costs. NASA provided the GAO access to detailed information on the technical, 
programmatic and cost elements of the RTF activities over the course of the investigation this 
past summer. NASA stands behind the cost estimates provided for the RTF efforts as known at 
that time. As the activity matures, so will the cost estimates. Comments on specific statements 
contained in the report are offered in the following paragraphs: 


Page 11: “NASA provided us with PRCB directives and in some cases, attachments which the 
Agency believes support the estimate. However, we did not find this support to be sufficient. 
According to NASA’s cost-estimating handbook, estimates should be documented with 
sufficient detail to be reproducible by an independent analyst.” 


NASA Response: In July 2004, NASA reported revised cost estimates for Shuttle return to flight 
(RTE) and advised Congress that the estimates will continue to evolve as RTF technical solutions 
matured. Indeed, multiple paths were being pursued for some of the technical solutions that 
greatly complicated the Agency’s ability to accurately forecast costs. The Agency also stated 
that the final costs for RTF would not be known until after the first Shuttle missions, at which 
time the effectiveness of the solutions could be properly assessed. NASA also stated that the 
cost estimates would be refined over the summer and another update would be provided to 
Congress in the fall of 2004. The GAO audit began in March of 2004 and completed in 
September of 2004. The auditors were advised that the estimates were still being refined as 
multiple-path solutions were being down-selected and the Agency began to finalize its RTF plan. 
NASA concedes that the estimates were not refined to the level that could withstand an 
independent audit at the time the GAO reviewed them, and that the Shuttle program is only now 
beginning to close the outstanding technical issues that must be solved to return to flight safely, 
and that allow for cost uncertainties to be resolved. 


Page 11: “In addition, our review of the documents indicated and NASA confirmed that quite a 
few of the estimates were based on undefinitized contract actions (UCAs) — that is, unnegotiated 
contract changes . .. As we have stated in our high-risk series, relying on unnegotiated changes 
is risky because it increases the potential for unanticipated cost growth.” 


NASA Response: The use of UCAs to advance RTF activity is both reasonable and necessary. 
As previously stated, the technical solutions to RTF recommendations put forth by the Columbia 
Accident and Investigation Board (CAIB) have evolved over time. NASA is breaking new 
ground with changes to the Shuttle’s external tank, thermal protection system, debris prevention 
and monitoring, and on-orbit inspection and repair techniques. In some cases, multiple paths 
have been chosen to ensure that at least one solution would be successful in achieving a safe 
return to flight. To facilitate the work, much of which is outside the scope of existing contracts, 
UCAs have been employed to avoid the months of delay that would result if NASA were to wait 
until its contractors responded to a quote request with a firm, auditable proposal. To maintain 
control over content and costs, NASA has used UCAs only when necessary, and strict limits 
have been placed on the scope of work and on “not-to-exceed” costs. This approach has 
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Appendix II: Comments from the National Aeronautics and Space Administration 








provided the Shuttle program the flexibility to pursue RTF efforts aggressively, but within 
reasonable and prudent controls. 


Page 12: “In June 2004, NASA established additional requirements for funding requests 
submitted to the [Shuttle program] PRCB. Under the new policy, an independent cost estimate 
must be developed for requests greater than $25 million, and a program-level cost evaluation 
must be completed for requests over $1 million... NASA provided us two examples ... Both 
examples had better support than those with PRCB directives, but documentary support was still 
not apparent.” 


NASA Response: NASA agrees that cost estimates for significant development activities should 
be appropriately documented. The additional requirements for independent cost estimates and 
internal controls established by the Shuttle program this past June, and acknowledged by the 
GAO audit, represent a significant step in ensuring that the appropriate level of documentation is 
developed as RTF technical solutions are determined, and made available for future examination. 


In summary, NASA believes that the support for the cost estimates provided for both the 
reinstatement of a Shuttle-based Hubble Servicing Mission and the implementation of the CAIB 
recommendations are accurate given the maturity of the design solutions for each these activities. 


Cordially, 


Frederick Gregory 
Deputy Administrator 
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